#training efficiency

‘Selective training on high-entropy tokens in LLMs improves reasoning performance and reduces computational costs, setting new benchmarks on AIME tests.’

Microsoft researchers demonstrate that the Muon optimizer drastically speeds up grokking in Transformer models, enabling faster transition from memorization to generalization compared to AdamW.